Providing representations of a video program with multiple video streams having different stream types

ABSTRACT

Methods and systems for the efficient and non-redundant transmission of a single video program in multiple frame rates, optionally employing a combination of video coding standards, in a way that is backwards-compatible with legacy receivers only supportive of some subsection of frame rates or of some subsection of video coding standards.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/342,569, filed Dec. 23, 2008, issued on Feb. 16, 2016 as U.S. Pat.No. 9,264,766 which is a continuation of U.S. patent application Ser.No. 11/132,060, filed May 18, 2005, now abandoned, which are entirelyincorporated herein by reference.

TECHNICAL FIELD

The present invention relates to digital television and, morespecifically to receivers with different capabilities for receiving,processing and displaying the same emission of a compressed videosignal, each receiver providing one in a plurality of picture formatsaccording to its respective capability.

BACKGROUND OF THE INVENTION

There are many different digital television compressed video pictureformats, some of which are HD. HDTV currently has the highest digitaltelevision spatial resolution available. The picture formats currentlyused in HDTV are 1280.times.720 pixels progressive, 1920.times.1080pixels interlaced, and 1920.times.1080 pixels progressive. These pictureformats are more commonly referred to as 720P, 1080i and 1080P,respectively. The 1080i format, which comprises of interlaced pictures,each picture or frame being two fields, shows 30 frames per second andit is deemed as the MPEG-2 video format requiring the most severeconsumption of processing resources. The 1080P format shows 60 framesper second, each frame being a progressive picture, and results in adoubling of the most severe consumption of processing resources. Areceiver capable of processing a maximum of 1080i-60 is also capable ofprocessing a maximum 1080P-30. However, broadcasters intend to introduce1080P-60 emissions and CE manufacturers intend to provide HDTVs and HDTVmonitors capable of rendering 1080P-60, in the near future. 1080P-60includes twice as much picture data as either 1080i-60 or 1080P-30. Dualcarrying channels or programs as 1080P-60 and 1080i-60 would not be anacceptable solution because it triples the channel consumption of asingle 1080i-60 transmission.

Therefore, there is a need for encoding 1080P-60 video for transmissionin a way that facilitates the superior picture quality benefits of a1080P-60 signal to 1080P-60 capable receivers while simultaneouslyenabling legacy 1080i-60 capable receivers to fulfill the equivalent ofa 1080P-30 signal from the transmitted 1080P-60 signal.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level block diagram depicting a non-limiting example ofa subscriber television system.

FIG. 2 is a block diagram of a DHCT in accordance with one embodiment ofthe present invention.

FIG. 3 illustrates program specific information (PSI) of a programhaving elementary streams including encoded video streams which may becombined to form a single video stream encoded as 1080P-60.

FIG. 4A illustrates first and second video streams in display order.

FIG. 4B illustrates pictures according to picture types in displayorder.

FIG. 4C illustrates transmission order of the pictures in display orderof FIG. 2B.

DETAILED DESCRIPTION

The present invention will be described more fully hereinafter withreference to the accompanying drawings in which like numerals representlike elements throughout the several figures, and in which an exemplaryembodiment of the invention is shown. This invention may, however, beembodied in many different forms and should not be construed as beinglimited to the embodiments set forth herein; rather, the embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the scope of the invention to those skilled in the art. Thepresent invention is described more fully hereinbelow.

It is noted that “picture” is used throughout this specification as onefrom a sequence of pictures that constitutes video, or digital video, inone of any of a plurality of forms. Furthermore, in this specification a“frame” means a picture, either as a full progressive picture or inreference to a whole instance of a full frame comprising both fields ofan interlaced picture.

Video Decoder in Receiver

FIG. 1 is a block diagram depicting a non-limiting example of asubscriber television system (STS) 100. In this example, the STS 100includes a headend 110 and a DHCT 200 that are coupled via a network130. The DHCT 200 is typically situated at a user's residence or placeof business and may be a stand-alone unit or integrated into anotherdevice such as, for example, the display device 140 or a personalcomputer (not shown). The DHCT 200 receives signals (video, audio and/orother data) including, for example, MPEG-2 streams, among others, fromthe headend 110 through the network 130 and provides any reverseinformation to the headend 110 through the network 130. The network 130may be any suitable means for communicating television services dataincluding, for example, a cable television network or a satellitetelevision network, among others. The headend 110 may include one ormore server devices (not shown) for providing video, audio, and textualdata to client devices such as DHCT 200. Television services areprovided via the display device 140 which is typically a television set.However, the display device 140 may also be any other device capable ofdisplaying video images including, for example, a computer monitor.

FIG. 2 is a block diagram illustrating selected components of a DHCT 200in accordance with one embodiment of the present invention. It will beunderstood that the DHCT 200 shown in FIG. 2 is merely illustrative andshould not be construed as implying any limitations upon the scope ofthe preferred embodiments of the invention. For example, in anotherembodiment, the DHCT 200 may have fewer, additional, and/or differentcomponents than illustrated in FIG. 2. A DHCT 200 is typically situatedat a user's residence or place of business and may be a stand alone unitor integrated into another device such as, for example, a television setor a personal computer. The DHCT 200 preferably includes acommunications interface 242 for receiving signals (video, audio and/orother data) from the headend 110 through the network 130 (FIG. 1) andfor providing any reverse information to the headend 110.

DHCT 200 is referred to as a receiver such as receiver 200 throughoutthis specification. The DHCT 200 further preferably includes at leastone processor 244 for controlling operations of the DHCT 200, an outputsystem 248 for driving the television display 140, and a tuner system245 for tuning to a particular television channel or frequency and forsending and receiving various types of data to/from the headend 110. TheDHCT 200 may, in another embodiment, include multiple tuners forreceiving downloaded (or transmitted) data. Tuner system 245 can selectfrom a plurality of transmission signals provided by the subscribertelevision system 100, including a 1080P-60 program. Tuner system 245enables the DHCT 200 to tune to downstream media and data transmissions,thereby allowing a user to receive digital media content such as a1080P-60 program via the subscriber television system. The tuner system245 includes, in one implementation, an out-of-band tuner forbi-directional quadrature phase shift keying (QPSK) data communicationand a quadrature amplitude modulation (QAM) tuner (in band) forreceiving television signals. Additionally, a user command interface 246receives externally-generated user inputs or commands from an inputdevice such as, for example, a remote control. User inputs could bealternatively received via communication port 274.

The DHCT 200 may include one or more wireless or wired interfaces, alsocalled communication ports 274, for receiving and/or transmitting datato other devices. For instance, the DHCT 200 may feature USB (UniversalSerial Bus), Ethernet, IEEE-1394, serial, and/or parallel ports, etc.DHCT 200 may also include an analog video input port for receivinganalog video signals. User input may be provided via an input devicesuch as, for example, a hand-held remote control device or a keyboard.

The DHCT 200 includes signal processing system 214, which comprises ademodulating system 213 and a transport demultiplexing and parsingsystem 215 (herein demultiplexing system) for processing broadcast mediacontent and/or data. One or more of the components of the signalprocessing system 214 can be implemented with software, a combination ofsoftware and hardware, or preferably in hardware. Demodulating system213 comprises functionality for demodulating analog or digitaltransmission signals. For instance, demodulating system 213 candemodulate a digital transmission signal in a carrier frequency that wasmodulated, among others, as a QAM-modulated signal. When tuned to acarrier frequency corresponding to an analog TV signal, demultiplexingsystem 215 is bypassed and the demodulated analog TV signal that isoutput by demodulating system 213 is instead routed to analog videodecoder 216. Analog video decoder 216 converts the analog TV signal intoa sequence of digitized pictures and their respective digitized audio.The analog TV decoder 216 and other analog video signal components maynot exist in receivers or DHCTs that do not process analog video or TVchannels.

A compression engine in the headend processes a sequence of 1080P-60pictures and associated digitized audio and converts them intocompressed video and audio streams, respectively. The compressed videoand audio streams are produced in accordance with the syntax andsemantics of a designated audio and video coding method, such as, forexample, MPEG-2, so that they can be interpreted by video decoder 223and audio decoder 225 for decompression and reconstruction aftertransmission of the two video streams corresponding to the 1080P-60compressed signal. Each compressed stream consists of a sequence of datapackets containing a header and a payload. Each header contains a uniquepacket identification code, or packet identifier (PID) as is the caseinMPEG-2 Transport specification, associated with the respectivecompressed stream. The compression engine or a multiplexer at theheadend multiplexes the first and second video streams into a transportstream, such as an MPEG-2 transport stream.

Video decoder 223 may be capable of decoding a first compressed videostream encoded according to a first video specification and a secondcompressed video stream encoded according to a second videospecification that is different than the first video specification.Video decoder 223 may comprise of two different video decoders, eachrespectively designated to decode a compressed video stream according tothe respective video specification.

Parsing capabilities 215 within signal processing 214 allow forinterpretation of sequence and picture headers. The packetizedcompressed streams can be output by signal processing 214 and presentedas input to media engine 222 for decompression by video decoder 223 andaudio decoder 225 for subsequent output to the display device 140 (FIG.1).

Demultiplexing system 215 can include MPEG-2 transport demultiplexing.When tuned to carrier frequencies carrying a digital transmissionsignal, demultiplexing system 215 enables the separation of packets ofdata, corresponding to the desired video streams, for furtherprocessing. Concurrently, demultiplexing system 215 precludes furtherprocessing of packets in the multiplexed transport stream that areirrelevant or not desired such as, for example in a 1080i-60 capablereceiver, packets of data corresponding to the second video stream ofthe 1080P-60 program.

The components of signal processing system 214 are preferably capable ofQAM demodulation, forward error correction, demultiplexing MPEG-2transport streams, and parsing packetized elementary streams andelementary streams. The signal processing system 214 furthercommunicates with processor 244 via interrupt and messaging capabilitiesof DHCT 200.

The components of signal processing system 214 are further capable ofperforming PID filtering to reject packetized data associated withprograms or services that are not requested by a user or unauthorized toDHCT 200, such rejection being performed according to the PID value ofthe packetized streams. PID filtering is performed according to valuesfor the filters under the control of processor 244. PID filtering allowsfor one or more desired and authorized programs and/or services topenetrate into DHCT 200 for processing and presentation. PID filteringis further effected to allow one or more desired packetized streamscorresponding to a program (e.g., a 1080P.sub.-60 program) to penetrateDHCT 200 for processing, while simultaneously rejecting one or moredifferent packetized stream also corresponding to the same program.Processor 244 determines values for one or more PIDS to allow topenetrate, or to reject, from received information such as tablescarrying PID values as described later in this specification. In analternate embodiment, undesirable video streams of a program are allowedto penetrate into DHCT 200 but disregarded by video decoder 223.

A compressed video stream corresponding to a tuned carrier frequencycarrying a digital transmission signal can be output as a transportstream by signal processing 214 and presented as input for storage instorage device 273 via interface 275. The packetized compressed streamscan be also output by signal processing system 214 and presented asinput to media engine 222 for decompression by the video decoder 223 andaudio decoder 225.

One having ordinary skill in the art will appreciate that signalprocessing system 214 may include other components not shown, includingmemory, decryptors, samplers, digitizers (e.g. analog-to-digitalconverters), and multiplexers, among others. Further, other embodimentswill be understood, by those having ordinary skill in the art, to bewithin the scope of the preferred embodiments of the present invention.For example, analog signals (e.g., NTSC) may bypass one or more elementsof the signal processing system 214 and may be forwarded directly to theoutput system 248. Outputs presented at corresponding next-stage inputsfor the aforementioned signal processing flow may be connected viaaccessible memory 252 in which an outputting device stores the outputdata and from which an inputting device retrieves it. Outputting andinputting devices may include analog video decoder 216, media engine222, signal processing system 214, and components or sub-componentsthereof. It will be understood by those having ordinary skill in the artthat components of signal processing system 214 can be spatially locatedin different areas of the DHCT 200.

In one embodiment of the invention, a first and second tuners andrespective first and second demodulating systems 213, demultiplexingsystems 215, and signal processing systems 214 may simultaneouslyreceive and process the first and second video streams of a 1080P-60program, respectively. Alternatively, a single demodulating system 213,a single demultiplexing system 215, and a single signal processingsystem 214, each with sufficient processing capabilities may be used toprocess the first and second video streams in a 1080P-60 capablereceiver.

The DHCT 200 may include at least one storage device 273 for storingvideo streams received by the DHCT 200. A PVR application 277, incooperation with the operating system 253 and the device driver 211,effects, among other functions, read and/or write operations to thestorage device 273. The device driver 211 is a software modulepreferably resident in the operating system 253. The device driver 211,under management of the operating system 253, communicates with thestorage device controller 279 to provide the operating instructions forthe storage device 273. Storage device 273 could be internal to DHCT200, coupled to a common bus 205 through a communication interface 275.

Received first and second video streams are deposited transferred toDRAM 252, and then processed for playback according to mechanisms thatwould be understood by those having ordinary skill in the art. In someembodiments, the video streams are retrieved and routed from the harddisk 201 to the digital video decoder 223 and digital audio decoder 225simultaneously, and then further processed for subsequent presentationvia the display device 140.

Compressed pictures in the second video stream may be compressedindependent of reconstructed pictures in the first video stream. On theother hand, an aspect of the invention is that pictures in the secondvideo stream, although compressed according to a second videospecification that is different to the first video specification, candepend on decompressed and reconstructed pictures in the first videostream for their own decompression and reconstruction.

Examples of dependent pictures are predicted pictures that reference atmost one picture (from a set of at least one reconstructed picture) foreach of its sub-blocks or macroblocks to effect its own reconstruction.That is, predicted pictures in the second video stream, can possiblydepend one or more referenced pictures in the first video stream.

Bi-predicted pictures (B-pictures) can reference at most two picturesfrom a set of reconstructed pictures for reconstruction of each of itssub-blocks or macroblocks to effect their own reconstruction.

In one embodiment, pictures in the second video stream referencedecompressed and reconstructed pictures (i.e., reference pictures) fromthe first video stream. In another embodiment, pictures in the secondvideo stream employ reference pictures from both the first and secondvideo streams. In yet another embodiment, a first type of picture in thesecond video stream references decompressed pictures from the secondvideo stream and a second type of picture references decompressedpictures from the first video stream.

Enabling Receivers with Different Capabilities

The present invention includes several methods based on two separatevideo streams assigned to a program rather than a single stream withinherent built-in temporal scalability. Existing receivers capable ofprocessing 1080i-60 video streams today would be deemed “legacy HDreceivers” at the time that broadcasters start emissions of 1080P-60programs. If a 1080P-60 program was transmitted without the advantage ofthis invention the “then” legacy HD receivers would not know how toprocess a 1080P-60 video stream, nor be capable of parsing the videostream to extract a 1080P-30 signal from the received 1080P-60. Thelegacy HD receivers were not designed to identify and discard picturesfrom a single 1080P-60 video stream. Furthermore, 1080P-60 in thestandard bodies is specified for a 1080P-60 receiver without backwardcompatibility to 1080i-60 receivers.

This invention enables 1080i-60 receivers to process the portion of the1080P-60 program corresponding to a first video stream and reject acomplementary second video stream based on PID filtering. Thus, byprocessing the first video stream, a 1080i-60 receiver provides aportion of the 1080P-60 program that is equivalent to 1080P-30. Theinvention is equally applicable, for example, to 1080P-50, assigning twoseparate video streams to a program. Future 1080P50-capable receiversprocess the 1080P-50 video from the two separate video streams accordingto the invention, while legacy 1080i-50-capable receivers process a1080P-25 portion of the 1080P-50 video program.

Hereinafter, 1080P-60 is used for simplicity to refer to a picturesequence with twice the picture rate of a progressive 1080P-30 picturesequence, or to a picture sequence with twice the amount of pictureelements as an interlaced picture sequence displayed as fields ratherthan full frames. However, it should be understood that the invention isapplicable to any pair of video formats with the same picture spatialresolution, in which a first video format has twice the “picture rate”of the second. The invention is also applicable to any pair of videoformats with the same picture spatial resolution, in which a first videoformat has “progressive picture rate” and the second has an “interlaced”or field picture rate, the first video format resulting in twice thenumber of processed or displayed pixels per second. The invention isfurther applicable to any two video formats in which the first videoformat's picture rate is an integer number times that of the secondvideo format or in which the number of pixels of a first video formatdivided by the number of pixels of a second video format is an integernumber.

Stream Types and Unique PIDs

The MPEG-2 Transport specification referred to in this invention isdescribed in the two documents: ISO/IEC 13818-1:2000 (E), InternationalStandard, Information technology—Generic coding of moving pictures andassociated audio information: Systems, and ISO/IEC 13818-1/Amd. 3: 2003Amendment 3: Transport of AVC video data over ITU-T Rec. H.222.0|ISO/IEC 13818-1 streams.

In accordance with MPEG-2 Transport syntax, a multiplexed transportcarries Program Specific Information (PSI) that includes the ProgramAssociation Table (PAT) and the Program Map Table (PMT). Informationrequired to identify and extract a PMT from the multiplexed transportstream is transmitted in the PAT. The PAT carries the program number andpacket_identifier (PID) corresponding to each of a plurality ofprograms, at least one such program's video being transmitted as encoded1080P-60 video according to the invention.

As shown in the FIG. 3, the PMT corresponding to a 1080P-60 programcarries two video streams, each uniquely identified by a correspondingPID. The first video stream in the PMT has a unique corresponding PID341 and the second video stream has its unique corresponding PID 342,for example. Likewise, the first and second video streams of the1080P-60 program have corresponding stream type values. A stream type istypically a byte. The stream type value for the first and second videostreams are video_type1 and video_type2, respectively.

In one embodiment, the stream type value, video_type1 equalsvideo_type2, therefore, both video streams are encoded according to thesyntax and semantics of the same video specification (e.g., both asMPEG-2 video or as MPEG-4 AVC). A receiver is then able to identify anddifferentiate between the first video stream and the second video streamby their PID values and the relationship of the two PID values. Forexample, the lower PID value of video_type1 would be associated with thefirst video stream. However, legacy HD receivers would not be able toincorporate such a processing step as a feature. However, there may betwo types of legacy receivers. During a first era, legacy receivers maybe HD receivers that are capable of processing a first video streamencoded according to the MPEG-2 video specification described in ISO/IEC13818-2:2000 (E), International Standard, Information technology—Genericcoding of moving pictures and associated audio information: Video. Thesecond video stream would likely be encoded with a video specificationthat provides superior compression performance, for example, MPEG-4 AVCas described by the three documents: ISO/IEC 14496-10 (ITU-T H.264),International Standard (2003), Advanced video coding for genericaudiovisual services; ISO/IEC 14496-10/Cor. 1: 2004 TechnicalCorrigendum 1; and ISO/IEC 14496-10/Amd. 1, 2004, Advanced Video CodingAMENDMENT 1: AVC fidelity range extensions. A second era, on the otherhand, may comprise legacy HD receivers that are capable of processing1080i-60 video encoded according to the MPEG-4 AVC specification.Because the latter legacy receivers have yet to be deployed, thesereceivers could be designed to support identification of the first videostream in a multiple video stream program from the lowest PID valuecorresponding to video_type1 in the PMT. Alternatively, the first videoentry in the PMT table, regardless of its PID value, would be consideredthe first video stream.

In another alternate embodiment, the streams are encoded according todifferent video specifications and the values of video_type1 andvideo_type2 in the PMT differ. For example, the first video stream wouldbe encoded and identified as MPEG-2 video in the PMT by a video_type1value that corresponds to MPEG-2 video. The second video stream would beencoded with MPEG-4 AVC and identified by a video_type2 valuecorresponding to MPEG-4 AVC.

In yet another alternate embodiment, video_type2 corresponds to a streamtype specifically designated to specify the complementary video stream(i.e, the second video stream of a 1080P-60 program). Both video streamscould be encoded according to the syntax and semantics of the same videospecification (e.g, with MPEG-4 AVC) or with different videospecifications. Thus, while the values of video_type1 and video_type2are different in the PMT table for a 1080P-60 program, both videostreams composing the 1080P-60 program could adhere to the same videospecification. Thus, video_type1's value identifies the videospecification used to encode the first video stream, but video_type2'svalue identifies both:

(1) the video stream that corresponds to the second video stream of the1080P-60 program, and

(2) the video specification (or video coding format) used to encode thesecond video stream.

A first video_type2 value then corresponds to a stream type associatedwith the second stream of a 1080P-60 program that is encoded accordingto the MPEG-2 video specification. A second video_type2 valuecorresponds to a stream type associated with the second stream of a1080P-60 program that is encoded according to the MPEG-4 AVCspecification. Likewise, other video_type2 values can correspond torespective stream types, each associated with the second stream of a1080P-60 program and encoded according to a respective video codingspecification.

In yet another novel aspect of the invention, when video_type2 does notequal video_type1 and their values signify different videospecifications, pictures in the second stream can still usereconstructed pictures from the first video stream as referencepictures.

Transmission Order of Pictures

Encoded pictures in the first and second video streams are multiplexedin the transport multiplex according to a defined sequence that allows asingle video decoder in a 1080P-60 receiver to receive and decode thepictures sequentially as if the pictures were transmitted in a singlevideo stream. However, because they are two separate video streams, a1080i-60 receiver can reject transport packets belonging to the secondvideo stream and allow video packets corresponding to the first videostream to penetrate into its memory to process a portion equal to1080P-30 video. Encoded pictures in the first video stream aretransmitted in transmission order, adhering to the timing requirementand bit-buffer management policies required for a decoder to process thefirst video stream as a 1080P-30 encoded video signal.

In one embodiment of the invention, FIG. 4A depicts the first and secondvideo streams in display order. P represents a picture and not a type ofpicture. Pi is the ith picture in display order. In a 1080P-60 receiver,the blank squares represent gaps of when the picture being displayed isfrom the complementary video stream. The width of a blank square is one“picture display” time. Non-blank squares represent the time interval inwhich the corresponding picture is being displayed.

Still referring to FIG. 4A, in a 1080i-60 receiver, a 1080P-30 picturecorresponding to the first video stream is displayed and the width oftwo squares represents the picture display time. Video stream 1 isspecified as 30 Hertz in alternating 60 Hertz intervals that correspondto even integers. Video stream 2 is specified as 30 Hertz in alternating60 Hertz intervals that correspond to odd integers.

FIG. 4B depicts pictures according to picture types in display order. Nisignifies the ith Picture in display order, where N is the type ofpicture designated by the letter I, P or B. In one embodiment, all thepictures in video stream 2 are B pictures and the 1080P-60 receiver usesdecoded pictures from video stream 1 as reference pictures toreconstruct the B pictures.

FIG. 4C corresponds to the transmission order of the pictures in displayorder in FIG. 4B. Each picture is transmitted (and thus received by thereceiver) at least one 60 Hz interval prior to its designated displaytime. I pictures are displayed six 60 Hz interval after being receivedand decoded. I pictures are thus transmitted at least seven 60 Hzintervals prior to its corresponding display time. The arrows from FIG.4C to FIG. 4B reflect the relationship of the pictures' transmissionorder to their display order.

Blank squares in FIG. 4C represent gaps when no picture data istransmitted for the respective video stream. The width of a blank squarecan be approximately one “picture display” time. Non-blank squaresrepresent the time interval in which the corresponding picture istransmitted. One or more smaller transmission gaps of no datatransmission may exist within the time interval in which a picture istransmitted. In essence, video stream 1 and video stream 2 aremultiplexed at the emission point in a way to effect the transmissionorder reflected in FIG. 4C and transmission time relationship depictedin FIG. 4C.

Bit-Buffer Management

A sequence of video pictures is presented at an encoder for compressionand production of a compressed 1080P-60 program. Every other picture isreferred as an N picture and every subsequent picture as an N+1 picture.The sequence of all the N pictures is the first video stream of the1080P-60 program and the sequence all the N+1 pictures is the secondvideo stream.

A video encoder produces the first video stream according to a firstvideo specification (e.g., MPEG-2 video) and the second video streamaccording to a second video specification (e.g., MPEG-4 AVC). In oneembodiment the second video specification is different than the firstvideo specification. In an alternate embodiment, the first and secondvideo specifications are the same (e.g., MPEG-4 AVC).

The video encoder produces compressed pictures for the first videostream by depositing the compressed pictures into a first bit-buffer inmemory, such memory being coupled to the encoder. Depositing ofcompressed pictures into the first bit-buffer is according to the buffermanagement policy (or policies) of the first video specification. Thefirst bit-buffer is read for transmission by the video encoder in oneembodiment. In an alternate embodiment, a multiplexer or transmitterreads the compressed pictures out of the first bit-buffer. The readportions of the first bit buffer are packetized and transmittedaccording to a transport stream specifications such as MPEG-2 transport.

Furthermore, the video encoder, the multiplexer, or the transmitter, orthe entity performing the first bit-buffer reading and packetization ofthe compressed pictures, prepends a first PID to packets belonging tothe first video stream. The packetized first video stream is thentransmitted via a first transmission channel.

Similarly, the second video stream is produced by the video encoder anddeposited into the first bit buffer. The second video stream is readfrom the first bit-buffer by the entity performing the packetization,and the entity prepends a second PID to packets belonging to the secondvideo stream, and the transport packets are transmitted via a firsttransmission channel.

In an alternate embodiment, the second video stream is produced by thevideo encoder and deposited into a second bit buffer. The entityperforming the packetization reads the second video stream from thesecond bit buffer and prepends the second PID to packets belonging tothe second video stream. The packetized second video stream is thentransmitted via a first transmission channel.

Both first and second video streams are packetized according to atransport stream specification, such as MPEG-2 Transport. Packetsbelonging to the second video stream are thus identifiable by a 1080P-60capable receiver and become capable of being rejected by a receiver thatis not capable of processing 1080P-60 programs.

The bit buffer management policies of depositing compressed picture datainto the first and/or second bit-buffers and reading (or drawing)compressed-picture data from the first and/or second bit-buffers, areaccording to the first video specification. These operations may befurther in accordance with bit-buffer management policies of thetransport stream specification. Furthermore, the bit-buffer managementpolicies implemented on the one or two bit-buffers may be according tothe second video specification rather than the first videospecification. In one embodiment, the first video stream's compresseddata in the bit-buffer is managed according to both: the bit buffermanagement policies of the first video specification and the transportstream specification, while the second video stream's compressed data inthe applicable bit-buffer is managed according to the bit buffermanagement policies of the second video specification as well as thetransport stream specification.

The bit-buffer management policies described above are applicable at theemission or transmission point in the network, such as by the encoderand the entity producing the multiplexing and/or transmission.Bit-buffer management policies, consistent with the actualimplementation at the emission or transmission point, are applicable atthe receiver to process the one or more received video streams of a1080P-60 program. The bit-buffer management policy implemented at theemission or transmission point may be provided to the receiver a priorifor each program (e.g., with metadata) or according to an agreed one ofthe alternatives described above that is employed indefinitely.

Enabling More than Two Receivers with Different Respective ProcessingCapabilities

In an alternate embodiment, the video encoder constitutes two videoencoders, a first video encoder producing the first video streamaccording to the first video specification, and a second video encoderproducing the second video stream, which is interspersed fortransmission in the transmission channel according to the pockets of “nodata” transmission of video stream 1 (as shown in FIG. 4C). The secondvideo encoder further producing the second video stream according to thesecond video specification.

In yet another embodiment, the process of alternating transmission ofcompressed pictures corresponding to the first video stream andcompressed pictures corresponding to the second video stream, results intransmission of a first set of consecutive compressed pictures fromdifferent the first video stream when it is the turn to transmit thefirst video stream, or a second set of consecutive compressed picturesfrom different the second video stream when it is the turn to transmitthe second video stream. For instance, instead of alternating betweenone compressed picture from the first video stream and one from thesecond video stream, two consecutive compressed pictures from the secondvideo stream may be transmitted after each transmission of a singlecompressed picture of the first video stream. Thus, a 1080P-90 Hertzprogram can be facilitated to 1080P-90 receivers and a 1080P-30 portionof the 1080P-90 program to 1080P-30 receivers. Furthermore, bypacketizing every second compressed picture in the second video streamwith a third PID value that is different than the first and second PIDs,three corresponding versions of the compressed 1080P-90 program arefacilitated respectively to a 1080P-30 receiver, a 1080P-60 receiver,and a 1080P-90 receiver, the latter being able to receive and fulfillthe full benefits of the 1080P-90 program.

In yet another embodiment, the number of consecutive compressed picturesthat is transmitted from the first video stream may be grater than one.For instance, if two consecutive compressed pictures from the firstvideo stream are transmitted and three compressed pictures from thesecond video stream are transmitted after transmission the two from thefirst video stream, a number of receivers with different processingcapabilities may be enabled. If two different PID values are employed, a1080P-50 receiver will receive a 1080P-50 Program and a 1080P-20receiver will receive a 1080P-20 corresponding portion. However, if fivedifferent PID values are used for the 1080P-50 program, five receivers,each with different processing capability will be capable of receiving aportion of the 1080P-50 program.

Third Video Specification

Headend 110 may receive from an interface to a different environment,such as from a satellite or a storage device, an already compressed1080P-60 program—a single video stream encoded according to a thirdvideo specification and according to a first stream specification. Thefirst stream specification may be a type of transport streamspecification suitable for transmission or a type of program streamspecification suitable for storage. The third video specification maycomprise of the first video specification, the second videospecification, or both the first and second video specificationsrespectively applied, for example, to every other compressed picture.However, the already compressed 1080P-60 program is received at headend110 encoded in such a way that it does not facilitate reception some ofits portions by receivers with processing capability that are less thanthose of a 1080P-60 receiver. In other words, it is received withoutinformation to inherent signal its different portions to receivers withdifferent processing capabilities.

Another novel aspect of this invention is that at least one from one ormore encoders, one or more multiplexers, or one or more processingentities at the point of transmission at headend 110, effectpacketization of the compressed pictures of the received 1080P-60program with a plurality of different PIDS, then transmitting the1080P-60 program as a plurality of identifiable video streams via thefirst transmission channel. Thus, headend 110 effects properpacketization and prepending of PID values to enable reception of atleast a portion of the 1080P-program to receivers with differentprocessing capabilities that are coupled to network 130.

The present invention includes methods and systems capable oftransmitting compressed video signals according to one or morecompression video formats, where compressed video signals correspond totelevision channels or television programs in any of a plurality ofpicture formats (i.e., picture spatial resolution and picture rate),including 1080i-60 and 1080P-60 formats. The compressed video signalswhich correspond to television channels or television programs in any ofa plurality of picture formats are received by a plurality of receivers,where each receiver may have a different maximum processing capability.Therefore, the present invention contemplates at least the followingcombinations for encoding, transmission and reception of video signals.In the following combinations of trio “input/receiver/display,” theinput, such as 1080P-60 input in the first combination instance, refersto a compressed video stream that is received at receiver 200 fromnetwork 130 via communication interface 242. The display, such as the1080P-60 Display in the first combination instance is a television, adisplay, or a monitor coupled to DHCT 200 via output system 248. TheDHCT 200 provides the compressed video stream corresponding to the“input” in “decoded and reconstructed” form (visible pictures) viaoutput system 248. The receiver, such as 1080P-60 Receiver in the firstcombination instance, refers to a receiver, such as DHCT 200, that hasthe processing capability specified in the trio.

1080P-60 Input/1080P-60 Receiver/1080P-60 Display

In order to process a 1080P-60 compressed video signal, a 1080P-60capable receiver receives a compressed 1080P-60 video stream via anetwork interface (or a communication interface). The 1080P-60compressed video signal is input by storing it in its memory and thereceiver decodes with a video decoder (or decompression engine) all thepictures corresponding to the 1080P-60 video signal (or compressed videostream). A 1080P-60 capable display is driven by all the decoded1080P-60 pictures.

1080i-60 Input/1080P-60 Receiver/1080P-60 Display

In order to process a 1080i-60 compressed video signal, the 1080P-60capable receiver receives a compressed 1080i-60 video stream via anetwork interface (or a communication interface). The 1080P-60compressed video signal is input by storing it in its memory and thereceiver decodes with a video decoder (or decompression engine) all thepictures corresponding to the compressed 1080i-60 video signal stored inmemory. The 1080P-60 receiver then deinterlaces the decoded 1080i-60signal with a de-interlacing algorithm based on information in two ormore 1080i fields, including a current 1080i field. The deinterlacingalgorithm makes decisions based on spatial picture information as wellas temporal information. The deinterlacing algorithm can further basedecisions on motion estimation or motion detection. A 1080P-60 capabledisplay is driven by all the decoded 1080P-60 pictures.

1080P-60 Input/1080P-60 Receiver/Non-1080P-60 Display

In order to process a 1080P-60 compressed video signal, the 1080P-60capable receiver receives a compressed 1080P-60 video stream via anetwork interface (or a communication interface). When driving anon-1080P-60 display, the receiver outputs a portion of all the decoded1080P-60 pictures or processes and scales the pictures of the decoded1080P-60 signal for display. When driving a non-1080P-60 display such asa 1080i-60 display, the 1080P-60 capable receiver could process a1080P-60 compressed video signal in full (as explained above) and output(or display) a portion of each of the decoded 1080P-60 pictures. Theportion may be a temporally-subsampled portion, a spatially-subsampledportion, or a portion resulting from a combination of atemporal-subsampling and spatially-subsampling. Alternatively, whendriving a non-1080P-60 capable display, the 1080P60-capable receiver isinformed by the user or through a discovery mechanism that the displayis not 1080P-60. Consequently, the 1080P-60-capable receiver can behaveas if it was a 1080P-30 receiver by not processing the second videostream.

1080i-60 Input/1080P-60 Receiver/Non-1080P-60 Display

When driving a non-1080P-60 display, a 1080P-60 receiver processes a1080i-60 compressed video signal and outputs the decoded 1080i-60pictures according to the picture format required to drive the non-1080display, processing and scaling the pictures of the decoded 1080i-60signal as required to drive the non-1080P-60 display.

1080P-60 Input/1080i-60 Receiver/Non-1080P-60 Display

In order to process a 1080P-60 compressed video signal, a 1080i-60capable receiver receives a compressed 1080P-60 video stream via anetwork interface (or a communication interface). The receiver inputs afirst portion of the 1080P-60 compressed video signal by storing it inmemory of receiver 200 and the receiver rejects a second andcomplementary portion of the 1080P compressed video signal byprohibiting it from penetrating any section, portion or buffer of itsmemory. The receiver 200 decodes with a video decoder (or decompressionengine) all the pictures corresponding to the first portion of the1080P-60 video signal; processing it as if it were a 1080i-60 compressedvideo signal. A 1080i-60 capable display is driven by the decoded firstportion of the 1080P-60 pictures.

1080P-60 Input/1080i-60 Receiver/1080P-60 Display—A

In order to process a 1080P-60 compressed video signal, a 1080i-60capable receiver receives a compressed 1080P-60 video stream via anetwork interface (or a communication interface). The receiver inputs afirst portion of the 1080P-60 compressed video signal corresponding to a1080i-60 compressed video signal by storing it in its memory and rejectsa second and complementary portion of the 1080P compressed video signalby prohibiting it from penetrating any section, portion or buffer of itsmemory. The receiver decodes with a video decoder (or decompressionengine) all the pictures corresponding to the first portion of the1080P-60 video signal, processing it as if it were a 1080i-60 compressedvideo signal. The receiver deinterlaces a decoded 1080i-60 signal with adeinterlacing algorithm based on information in two or more 1080ifields, including a current 1080i field. The deinterlacing algorithmmakes decisions based on spatial picture information as well as temporalinformation. The deinterlacing algorithm can further base decisions onmotion estimation or motion detection. A 1080P-60 capable display isdriven by all the decoded and deinterlaced 1080i-60 pictures as a1080P-60 signal.

1080P-60 Input/1080i-60 Receiver/1080P-60 Display—B

In order to process a 1080P-60 compressed video signal, a 1080i-60capable receiver receives a compressed 1080P-60 video stream via anetwork interface (or a communication interface). The receiver inputs afirst portion of the 1080P-60 compressed video signal corresponding to a1080i-60 compressed video signal by storing it in its memory and rejectsa second and complementary portion of the 1080P-60 compressed videosignal by prohibiting it from penetrating any section, portion or bufferof its memory. The receiver decodes with a video decoder (ordecompression engine) all the pictures corresponding to the firstportion of the 1080P-60 video signal, processing it as if it were a1080i-60 compressed video signal. In order to drive a 1080P-60 capabledisplay that is capable of receiving a 1080i-60 signal and internaldeinterlacing, the display is driven by all the pictures of the decoded1080i-60 compressed video signal as a 1080i-60 signal. The 1080P-60display deinterlaces the received 1080i-60 signals according to itsdeinterlacing capabilities.

Encoding and Transmission

The encoder produces a 1080P-60 encoded video stream according to avideo specification (i.e., MPEG-2 video or MPEG-4 AVC), and assigns afirst PID value to packets of every other encoded picture correspondingto the 1080P-60, and assigns a second PID value to every packet of thesubsequent picture to the “every other” picture just mentioned, wherethe second PID value is different from the first PID value. Denoting“every other picture” by N, every subsequent picture is then N+1; andthe first PID_value is used for N, while the second PID_value is usedfor N+1.

The encoder in one embodiment encodes all pictures according to a singlevideo format, e.g., MPEG-4 AVC, and adheres to the buffer model of thevideo specification. The encoder in a second embodiment encodes thepictures that correspond to N according to a first video specificationand in compliance with the video specification's buffering model, andaccording to a variable-bit rate model. The encoder further encodes thealternate pictures, every “N+1” picture, according to a second videospecification, the second video specification being different from thefirst video specification. These alternate pictures are encodedaccording to the syntax of the second video specification, but managedand transferred into a transmission buffer according to the first videospecification's buffering model. The encoder further employs in its“encoding loop” a model, or parts thereof, of a receiver's videodecoder, including reference pictures, in it's memory.

Encode 1080P at 60 frames per second, into a single output, ensuringthat every other picture (in both decode order and presentation order)is a non-reference picture. Every picture encoded is a progressive framerepresenting 1/60.sup.th seconds. Now, every other picture can beseparated into a new PID. This new PID may be called “PID B”, and theother PID may be called “PID A”. PID B contains only non-referencepictures that can optionally be included in the decoding of PID A. Inthis separation process, the original picture ordering must bemaintained within the multiplex. For example, a picture in one PID mustend before the next picture begins in the other PID.

For backwards-compatibility, the frame rate value in PID A should be setat 30 frames per second; and the temporal references in PID A should becorrected for the separated pictures; and as a convenience, the temporalreferences in PID B should be set to match those in PID A, such thateach picture pair shares a temporal reference number. The 1080P-60capable decoder will be aware that the frame rate is actually 60 framesper second, and will support the pairs of duplicate temporal references.When decoding both PID A and PID B in combination, the decoder shouldexpect two of every temporal reference number, adjacent in presentationorder. Therefore, for example, it can use the temporal reference numbersto detect a missing picture. Picture re-ordering within the decoder maybe based on the sequence of picture types received, as normal.

The following are examples of this scheme demonstrating how a decodercould receive PID A alone, or receive the combination of PID A and PIDB. In these examples, the “B”-type pictures represent non-referenceframes. Also, these examples are given in decode order, and the numbersrepresent temporal references (indicating presentation order).

EXAMPLE 1

IBBBP . . .

Before Temporal Reference Number (TRN) Correction:

PID A: I3_B1_P7_B5_P11_B9_P15_B13.sub.--

PID B: _B0_B2_B4_B6_B8_B10_B12_B14

After TRN Correction:

PID A: I1_B0_P3_B2_P5_B4_P7_B6.sub.--

PID B: _B_B1_B2_B3_B4_B5_B6_B7

EXAMPLE 2

IBP . . .

Before TRN Correction:

PID A: I1_P3_P5_P7_P9_P11_P13_P15.sub.--

PID B: _B0_B2_B4_B6_B8_B10_B12_B14

After TRN Correction:

PID A: I0_P1_P2_P3_P4_P5_P6_P7.sub.--

PID B: _B0_B1_B2_B3_B4_B5_B6_B7

In the PMT, PID B can be designated by a new stream_jype. A common setof audio streams may serve each case: 1) using only PID A 2) using bothPID A and PID B.

In the above described method of encoding and transmission, theseparation of every other frame occurred after encoding. In analternative embodiment, separation occurs prior to encoding. At oneencoder's input, supply every other frame of a 1080P-60 hz signal.Encode this as 1080P-30 hz. Simultaneously, supply another encodingprocess with the alternate frames, also at 1080P-30 hz. Presentationtime stamps (PTSs) shall be generated for every picture, referencing acommon clock. The result is two video streams, each being legitimate1080P-30 hz. A 1080P-60 capable decoder may decode both simultaneously,as a dual-decode operation, to be recombined in the display process.There need be no further correlation between the two PIDs than thecommonly referenced PTSs. For example, the group of pictures (GOP)structures, as defined by the video specification (e.g., MPEG-2 videoGOP) may be independent, and the buffering may be independent. Torecombine the dual 1080P-30 streams into a single 1080P-60 output, thedual-decoder's display process will choose decoded pictures to put ondisplay in order of PTS. If the picture for a particular time intervalhas not yet been decoded, possibly due to some data corruption or loss,then the previous picture will simply be repeated through that timeinterval. If any picture is decoded later than its PTS elapses, it is tobe discarded. Even though both PIDs may be completely independent,because they reference the same clock, there is no risk that a picturefrom one PID is sent later than the presentation time of a followingpicture from the other PID, as long as each PID's buffer is maintainedcompliantly within the multiplex.

PID B in the PMT may be designated by a new stream_type, which may beallocated by MPEG, or which may be a user-private stream_type thatindicates a privately managed stream. The new stream_type would not berecognized by legacy receivers, so the associated PID B would beignored. As an additional method of unambiguous identification of thespecial second PID, the registration descriptor may be used in theES_descriptor_loop of the PMT to register a unique and private attributefor association with PID B. Any combination of the above methods may beused, as deemed adequate and sensible. A common set of audio streams mayserve each case: 1) using only PID A 2) using both PID A and PID B. Themethods described above use a separate PID to carry additionalinformation. In those cases, the separate PID can optionally be ignoredby the decoder. In another alternative embodiment, a single video PIDmay be used to carry both the base information and the additionalinformation, while still providing a way to optionally reject theadditional information. A separate packetized elementary stream (PES) IDcan be used such that a new PMT descriptor, which would be allocated byMPEG, may designate one PES ID for the base layer, and a different PESID for the additional information, both carried by the same PID. In thisway, existing PES IDs may be identified as base, and supplemental,without the need for new PES IDs to be allocated. The decoder that needsonly the base layer may discard those PES packets whose ID does notmatch the ID designated as the base layer in the PMT. The decoder thatcan use both may simply not reject either. This approach is applicableto both schemes: post-encoding-separation and prior-encoding-separation.

The foregoing has broadly outlined some of the more pertinent aspectsand features of the present invention. These should be construed to bemerely illustrative of some of the more prominent features andapplications of the invention. Other beneficial results can be obtainedby applying the disclosed information in a different manner or bymodifying the disclosed embodiments. Accordingly, other aspects and amore comprehensive understanding of the invention may be obtained byreferring to the detailed description of the exemplary embodiments takenin conjunction with the accompanying drawings, in addition to the scopeof the invention defined by the claims.

We claim:
 1. A method, comprising: providing a program as a multiplexedplurality of streams (MPOS), the MPOS comprising a first video streamcoded according to a first video specification and a second video streamcoded according to a second video specification, the second video streamcomplementary to the first video stream, wherein the first video streamcomprises a first representation of the program and a combination of thefirst video stream and second video stream comprises a secondrepresentation of the program; providing association information for theprogram, the association information providing a plurality of streamassociations, wherein each respective stream association comprises astream type and a respectively corresponding identifier for packets ofthe stream in the MPOS (IFPOTSITM), wherein the stream type of a firststream association and a second stream association of the plurality ofstream associations correspond to video data, wherein a first streamtype value of the stream type for the first stream association isdifferent than a second stream type value of the stream type for thesecond stream association, wherein the stream type of the first streamassociation has a first unique packet identifier (PID) and the streamtype of the second stream association has a second unique PID, whereinthe IFPOTSITMs for the first stream association and the second streamassociation are different, wherein the first stream type valueidentifies the first video specification used to encode the first videostream, and wherein the second stream type value identifies both thefirst video stream which is complementary to the second video stream andthe second video specification used to encode the second video stream;and compressing a first type of picture corresponding to the secondvideo stream based on referencing decompressed pictures from the secondvideo stream and a second type of picture corresponding to the secondvideo stream based on referencing decompressed pictures from the firstvideo stream.
 2. The method of claim 1, further comprising compressingpictures corresponding to the second video stream based on reconstructedpictures of the second video stream, said compression independent ofreconstructed pictures of the first video stream.
 3. The method of claim1, further comprising compressing pictures corresponding to the secondvideo stream based exclusively on reconstructed reference pictures ofthe first video stream.
 4. The method of claim 3, wherein the firstvideo specification is different than the second video specification. 5.The method of claim 3, wherein the second video stream consists ofcompressed non-reference pictures.
 6. The method of claim 1, furthercomprising compressing pictures corresponding to the second video streambased on reconstructed reference pictures of the first and second videostreams.
 7. The method of claim 1, wherein the differences in valuesindicates that the first video specification is different than thesecond video specification, and further comprising compressing picturescorresponding to the second video stream based on reconstructedreference pictures of the first video stream.
 8. The method of claim 1,wherein providing further comprises multiplexing the first and secondvideo streams in a transport multiplex, wherein compressed pictures ofthe first video stream are interspersed with compressed pictures of thesecond video stream.
 9. The method of claim 8, wherein providing furthercomprises transmitting the compressed pictures of the first and secondvideo streams in transmission order.
 10. An apparatus, comprising: amemory; and one or more processors configured to: provide a program as amultiplexed plurality of streams (MPOS), the MPOS comprising a firstvideo stream coded according to a first video specification and a secondvideo stream coded according to a second video specification, the secondvideo stream complementary to the first video stream, wherein the firstvideo stream comprises a first representation of the program and acombination of the first and second video streams comprises a secondrepresentation of the program; and provide association information forthe program, the association information providing a plurality of streamassociations, wherein each respective stream association comprises astream type and a respectively corresponding identifier for packets ofthe stream in the MPOS (IFPOTSITM), wherein the stream type of a firststream association and a second stream association of the plurality ofstream associations correspond to video data, wherein a first streamtype value of the stream type for the first stream association isdifferent than a second stream type value of the stream type for thesecond stream association, wherein the stream type of the first streamassociation has a first unique packet identifier (PID) and the streamtype of the second stream association has a second unique PID, andwherein the IFPOTSITMs for the first stream association and the secondstream association are different, wherein the first stream type valueidentifies the first video specification used to encode the first videostream, and wherein the second stream type value identifies both thefirst video stream which is complementary to the second video stream andthe second video specification used to encode the second video stream;and compress a first type of picture corresponding to the second videostream based on referencing decompressed pictures from the second videostream and a second type of picture corresponding to the second videostream based on referencing decompressed pictures from the first videostream.
 11. The apparatus of claim 10, wherein the one or moreprocessors are further configured to compress pictures corresponding tothe second video stream based on reconstructed pictures of the secondvideo stream, said the compression independent of reconstructed picturesof the first video stream.
 12. The apparatus of claim 10, wherein theone or more processors are further configured to compress picturescorresponding to the second video stream based exclusively onreconstructed reference pictures of the first video stream.
 13. Theapparatus of claim 12, wherein the first video specification isdifferent than the second video specification.
 14. The apparatus ofclaim 12, wherein the second video stream consists of compressednon-reference pictures.
 15. The apparatus of claim 10, wherein the oneor more processors are further configured to compress picturescorresponding to the second video stream based on reconstructedreference pictures of the first and second video streams.
 16. Theapparatus of claim 10, wherein the differences in values indicates thatthe first video specification is different than the second videospecification, and wherein the one or more processors are furtherconfigured to compress pictures corresponding to the second video streambased on reconstructed reference pictures of the first video stream. 17.The apparatus of claim 10, wherein providing further comprisesmultiplexing the first and second video streams in a transportmultiplex, wherein compressed pictures of the first video stream areinterspersed with compressed pictures of the second video stream. 18.The apparatus of claim 17, wherein providing further comprisestransmitting the compressed pictures of the first and second videostreams in transmission order.